METHOD AND APPARATUS FOR AUTOMATED IMAGE 
CORRECTION FOR DIGITAL IMAGE ACQUISITION 



TECHNICAL FIELD 



The invention relates to methods for assuring correct colors in a digital image 
file as a picture is captured by a camera, scanned, or printed onto a medium 
at a specified printer location, or simply displayed in the digital space. More 
particularly, the invention relates to methods for transforming the colors in a 
digital image to a color corrected digital image. 



DESCRIPTION OF THE PRIOR ART 



The traditional method for creating photographic images was based upon the 
silver halide film process. A camera holds a film that contains silver halide 
particles which are sensitive to light. When the photographer takes a picture, 
the image is recorded on the silver halide particles, which are later chemically 
processed. To make prints, especially enlargements, the photographic image 
is then transferred via a second photographic process to paper containing a 
layer of silver halide particles, and the process is repeated. In the case of 
color photography, the film and the paper use silver halide for each color also 
but include tinted filters to assure that red light, for example, exposes one 
layer, blue light another layer, etc. 



Different light sources have different balances of energy. As is quite well 
known in optics, visible light is made up of different wavelengths of energy. 
Direct sunlight at nnidday is considered neutral; incandescent light is 
considered warm, because it has a higher proportion of red light; and 
fluorescent light is considered cool, because it has a higher proportion of 
green and blue light. 

Although the eye sees these differences in light balance, the brain generally 
compensates by adjusting the perceived color balance. Achieving a proper 
color balance in film and paper involves adjusting the filters used on the 
separate layers of the silver halide crystals. Balancing a specific print is a 
painstaking, trial and error process. 

Traditional still cameras do not affect color balance, except in the trivial sense 
that lenses may have very slight chromatic effects. The film and paper have 
color characteristics that significantly affect the color balance of the final print. 
Most color snapshots today are printed using highly automated electronically 
controlled printing machines that take these color profiles into account, and 
require little if any user intervention to get pleasing color output. 

However, these automated processes cannot completely compensate for all 
variables, because a large portion of color variability is attributable to the light 
source used to illuminate the subject. Different sources of light include varying 
levels of the components of the spectrum of visible light, which causes them 
to appear to have different overall color balances. Incandescent light tends to 
favor yellow; florescent light tends to favor bluish green. While the human 



brain tends to accommodate quickly to these differences and recalibrate the 
internal sense of what is a neutral white or gray, a photograph cannot 
completely compensate one situation into another and the results can be very 
obvious even to non-professionals. 

5 

Color matching in professional photography is often of critical importance. 
Color matching in professional photography is generally time consuming 
because colors of various items, such as clothing and other consumer goods, 
must be precisely reproduced for catalogs, advertisements and brochures. 
10 The process could, therefore, get prohibitively expensive in most cases unless 
a photographer with experience can expedite the processing. 

in the last few years, digital photography has begun mailing significant 
inroads as an alternative medium for creating and manipulating images. In 
15 addition, the availability of inexpensive scanners has made it possible for 
many people to take photographs created using traditional film processes and 
manipulate, transmit or print them digitally. 

Lately, digital color printing has also evolved significantly, In part, because of 
20 color profiles created to map the color characteristics of specific devices. It is, 
therefore, possible now to get consistent color output on a variety of devices. 
However, it Is considerably more difficult to assure proper color In the creation 
of a digital file, such as a digital image file provided by a digital camera. 

25 A common problem when one deals with digital color imaging devices is 
getting the colors on the image as printed or displayed on a monitor to match 



3 



the true object. For example, a color scanner as an Input device is associated 
with device dependent color space containing colors such as red, green, and 
blue (RGB). The device dependent color space, where colors are described in 
digital RGB values, determines the color representation of that particular 
5 device. This color space is different for each such imaging device and is thus 
called device dependent color space. 

Just as input devices are associated with device dependent color space, 
output devices, e.g. printers, also operate in device-dependent color space. 
10 Colors printed on any given printer typically do not match colors printed on a 
different printer because each printer is associated with a cyan, magenta, 
yellow, and biack (CMYK) device dependent color space. Each image- 
rendering device, such as a printer or a monitor, has a limited range of colors 
that it can reproduce, known as a gamut. 

15 

Under different lighting conditions, devices such as digital cameras may also 
have difficulty accurately capturing the full range of values of brightness or 
contrast. If the image does not accurately distinguish between darker values, 
then shadow detail is lost and pictures may appear muddy. If the image does 
20 not accurately distinguish between light areas, highlights appear without detail 
and the image may look washed out. By using a test image with a range of 
values of the same color, such as a grey scale, it is possible to adjust the 
range of contrast to achieve a more pleasing and natural color balance. 

25 
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It would be advantageous to provide digital color correction methods to 
correct for the differences between the reference colors in a color chart and 
those same colors as presented in an image that is recorded by a 
photographic device or a film medium. 

5 

It would also be advantageous to provide automated color correction methods 
to correct for the differences between the reference colors in a color chart and 
those same colors as presented in an image that is recorded by a 
photographic device or a film medium, and to adjust for optimum contrast and 
10 brightness. 

SUMMARY OF THE INVENTION 

The invention provides color corrected digital profiling methods that can 
15 correct for differences between reference colors according to a color chart 
and the reference colors as recorded and submitted as an image recorded by 
a photographic device or a film medium. In the presently preferred 
embodiments, this process is automated so that the time spent by a user of 
photo imaging devices is minimized and so that special expertise is not 
20 required to achieve excellent color fidelity and contrast. In one presently 
preferred embodiment, this automated process is encoded in software or 
hardware included within the camera or other image acquisition device. 

One method in accordance with the invention comprises acquiring a digital 
25 image file of the test image or target containing color samples of known 



values at known positions within the target using a digital image acquisition 
device such as a digital camera, scanner, or other suitable device. The 
method further involves comparing the digital color values of the test image 
with the known values to determine whether the test image is properly aligned 
and oriented. The method further involves comparing the known values of the 
colors in the test image with the actual values obtained by the device under 
the specific conditions in which it is being used. The method further Involves 
generating a corrective color profile that adjusts the hue, contrast, and overall 
exposure of the image to match the reference values as closely as possible. 
In one embodiment, this process takes place entirely within software and/or 
hardware contained within the image acquisition device. In another 
embodiment, the method involves use of software loaded on a computer such 
as a PC. fn a third embodiment, the method involves registering over a 
network such as the Internet to obtain access to profiling software. The 
registration itself involves using a password or access code for using the 
software on the computer. In the presently preferred embodiment, the test 
image is submitted to a server upon which the color correction software is 
stored. The profiling software itself checks the image file generated from the 
scanner or the digital camera and compares the values obtained with the 
known reference values of the colors on the target test sheet under specific 
lighting conditions. The profiling software specifically checks the submitted 
object file for orientation, alignment, color intensity, and hue with respect to 
the image of the target test sheet. 

In each of the preferred embodiments, an image transformation algorithm is 
created to match colors of the submitted image file with colors of the target 



test sheet and thereby creates a color corrected image profile for the 
submitted image file. 

The image transformation algorithm matches the colors of the submitted 
5 Image file with colors of the target test sheet using a binary color coding in 
which a unique binary number represents each color. Where the color 
correction process is contained within the image acquisition device, the image 
transformation algorithm is used to correct subsequent images taken under 
the same lighting conditions. Where the color correction process is in software 

10 in a local computer or on a network, the user can apply the corrective profile 
to other images selected by the user, the generated color corrected image 
can be manipulated, transmitted to a local or a remote printer, transmitted to a 
digital exhibit space, or simply compared to the submitted digital image. The 
generated color corrected image profile is typically transmitted in digital space 

15 via e-mail. 

The color corrected image profile can be saved for future use or uploaded to a 
Web site wherein several image files can be color corrected in sequence or in 
random order. 

20 

BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is an illustration of the components in a color matching technique 
according to the Invention; 

25 
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FIG. 2 is a flow diagram of the steps involved in profiling software for the 
formation of color corrected digital image files according to the invention; 

FIG. 3A is a flow diagram of the steps involved in obtaining digital color 
images using a color matching technique on a local printing machine 
according to the invention; 

FIG. 3B is a flow diagram of the steps involved in obtaining digital color 
images using a color matching technique on a remote printing machine 
according to the invention; 

FIG. 4A is a flow diagram of the steps involved in uploading images onto a 
digital exhibit space using a color matching technique according to the 
invention; and 

FIG. 4B is a flow diagram of the steps involved in obtaining access to color 
images on a digital exhibit space using a color matching technique according 
to the invention. 

DETAILED DESCRIPTION 

The invention provides a solution to the problem of color matching of digital 
images using a color matching technique involving a color chart for profiling, 
and profiling software that generates an image profile after correcting for the 
differences between the color chart of known hue, saturation, and intensity 
with a recorded image from, for example, a digital camera. The generated 



profile can be stored and viewed in an exhibit space or used for processing 
multiple images of color corrected digital files. 

To use the herein described color matching technique (FIG.1), the user is 
5 provided with a target test sheet 2 and an imaging device, such as a digital 
camera 4 or a color scanner 6. To generate a color profile for a digital camera 
4, the user takes a photograph of the target test sheet 2 under the specific 
lighting conditions for which the profile is desired. In the case of a scanner 6, 
the user scans the target test sheet 2. 

10 

The following steps are same for both the scanner 6 and the digital camera 4. 
The user then downloads the file containing an image of the target to a client 
computer 8. The user then transmits the file over a network such as the 
Internet 10 to a server 12. The server 12 then opens the file 14. 

15 

FIG. 2 is a flow diagram showing the steps involved in profiling software for 
forming color corrected digital image files. To use the profiling software in one 
embodiment of the invention, one must be registered and logged in to a Web 
site. Once the user is registered 22 and logged in, the user can access 
20 software that allows the user to apply the transformation algorithms available 
at the Web site. Ideally, the user is able to accomplish these transformations 
through a one-click process. After downloading the application, the user 
uploads a file 24, which the Web site logs with the user's unique identifier. 

25 The Web site then runs a program that uses a pattern recognition algorithm or 
algorithms 26 to assure that the image of the target captured by a 
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photographer is aligned 28 correctly so that, for example, the cyan target in 
the transmitted image is corrected to equal the original cyan target, rather 
than the blue or green, which would result in a major distortion of the color 
balance. There are two general approaches that could be taken to accomplish 
this. One approach is that, prior to submission of the image containing the 
target, the user software could prompt the user to identify via mouse clicks the 
location of the corners or other identifying locations in the targe that the 
software could use as a reference to align the image of the target and 
accurately compare the values of each color sample in the submitted image 
with the correct color image sample. 

A more automated system operates without requiring the user to take this 
step. In the case of a color correction system contained within the image 
acquisition device itself, the device, such as a camera, accomplishes this 
result without requiring the user to participate other than to inform the camera 
through an input, such as selection of a menu item or pressing of a button or 
other input device. The color correction software then samples the image, 
searching, for example for the 100% cyan region in the test image by 
determining whether there is a region in the image that is roughly the 
appropriate shape and with color values relatively close to cyan. The 
software samples a large number of regions to find a region that meets the 
selection criteria. Once the software identifies a region that appears to satisfy 
the selection criteria. The software identifies such a region, it searches the 
areas immediately adjacent to the identified region for a color sample known 
to be adjacent to the that color sample in the test image. If the software does 
not locate the required color sample candidate, it rejects the original sample 

10 



and starts the process again. If the software does locate the expected second 
color sample, It repeats the process In searching for a third known sample, 
and so on through sufficient iterations to achieve a sufficiently high level of 
confidence that the target has been properly located and oriented. Such 
image recognition and pattern recognition software are known in the art. 

The software then executes a transformation algorithm that compares the 
numerical value of specific color samples in the image captured by the user to 
the known values in the original target, and generates a numerical difference 
value. For example, in the CMYK space, the proper value for a cyan area in 
the target image might be 255, 0,0,0, where the first value represents the level 
of cyan on a scale of 0 to 255, the second the level of magenta, the third of 
yellow, and the last of black. If for example, the software determined that the 
value for the area as captured by the acquisition device was 250,10,2,5, the 
program would compare the submitted values to the known values and 
generate a transformation profile of +5, -10,-2, -5 to correct the color 
imbalance and thus match the color balance of the original image 30 as 
closely as possible. The program then creates a profile, which consists of the 
settings or adjustments needed by the software to convert an image with the 
color balance 32 in the submitted image to one having accurate color. 

In the case of software or hardware included within the image acquisition 
device, the device then uses the profile for subsequent images. The image 
acquisition device uses the new profile until the user inputs his desire to stop 
using it. Alternatively, the device could use additional software to sense that 
the lighting conditions have changed, discontinue use of the profile, and 



prompt the user to repeat the profiling process. In the case of image 
correction on a local computer or over a network, a file containing the profile is 
created and optionally saved. 



5 In the case of software on a local computer, the user can then apply the 
profile to one or more images. In the case of software operating over a 
network, the server then transmits the profile, either via email or by allowing 
the user to otherwise download the profile. In the network embodiment of the 
invention, the server 12 (FIG. 1) sends the profile back to the client computer 

10 8 over a network 1 0. The user can then use the software 1 6 to load the profile 
onto the client computer and apply the profile 14 to other image files, thus 
correcting color balance of those images. Ideally, this function is as simple as 
the user using a mouse or other cursor controller to drag the profile 14 icons 
onto an image file,or vice versa. The user then has the option to accept or 

15 reject the corrected Image and save the corrected file 14. This procedure can 
be repeated several times, for example at a customer's request, until 
terminated, E.G. herein disclosed by the customer 38 (FIG. 2). 

A photographer wishing to use the herein disclosed color matching technique 
20 first obtains a target test sheet 2 (FIG. 1), which may be a printed cardboard, 
plastic, paper, or other test sheet containing a color target (a test pattern of 
color samples), as well as basic instructions for use 50. He then uses his 
digital camera 4 to photograph the test sheet in the exact lighting conditions 
he wishes to control. Typically, this is accomplished by taking a test shot 
25 immediately before 54 the picture he desires to calibrate. In the case of 
software embedded in a digital camera, the user simply signals the camera to 



adjust exposure, and all subsequent photographs are automatically adjusted 
for the current lighting conditions. 

In the case of software available over the network, the photographer then 
registers with the server or website 60. The server or web site (FIG.2, 22) 
gives the user a unique account. The photographer then uploads the test 
photograph to the sen/er or web site 62. The user receives an executable file 
in the fornn of an email attachment or other file from the server or web site. 

By clicking on the executable file, the user can then create a profile on his 
computer that allows him to process images taken with that camera and 
correct for the lighting conditions. For example, the user could open the image 
using software such as Adobe Photoshop or a web browser. Clicking on the 
desired icon automatically starts a software process that executes 64 the 
image transformation and allows the user to compare 66 the original and 
corrected images side-by-side on the screen. The user can then choose 68 to 
accept 70 or reject 72 the corrected image and save it in a new file 74. If the 
user has local access to a calibrated printer 76, he can then print 78 the 
images using the profile. If the user wishes to print remotely (FIG. 3B), he can 
upload the profiled print 102 to the server or web site in encrypted form and 
then specify a remote location to print 104 the file as corrected by the 
algorithm herein. At the time the file is processed for printing, this step is 
followed by the step of applying a profiling of the target printer 106 and 
terminating the session 1 10 as soon as the printing is done 108. Printing may 
occur immediately after the file is sent to the remote location, or may be 
delayed, for example, until the user arrives at the remote printing location. 



Alternately, the user can upload the corrected image to a digital exhibit space 
linked to the server or web site (FIG. 4Aand 4B). The user has the option to 
set permission and/or passwords to control the access to the digital images. 
5 FIG. 4A is a flow diagram that shows the steps involved in uploading digital 
images onto a digital space using the herein disclosed color matching 
technique. 

The user starts the uploading process 150 by obtaining a target test sheet 152 
10 and photographs the test sheet 156 in specific lighting conditions. Such 
lighting conditions could include, for example, incandescent lighting, 
fluorescent lighting, daylight, or any combination thereof. The user then 
captures the image for which correction is desired under the same lighting 
conditions 158. 

15 

In the case of a camera containing the color correction software, the user then 
instructs the camera to optimize the image. Alternatively, the camera could 
apply the transformation as a filter prior to capturing the image, to maximize 
the physical capabilities of the device. 

20 

In the case of software accessible from a network, the user then loads 162 the 
test photograph to the sen/er or web site. The profiling software color matches 
the values for the color target as captured in the picture to the known values 
for those colors in the target test sheet from step 156. An executable file or, 
25 alternatively a profile to be used by profiling software for subsequent 
processing is created by the profile generating software and transmitted to the 



user via e-mail 164or other electronic means. The user can check one or 
more of his pictures for color matching by clicking or dragging the image onto 
the image transformation icon 166 or, alternatively, by dragging the icon 166 
onto the image. The user can compare the original picture image with the 
5 color corrected image on the screen 168 and check 170 to accept 174 or 
reject 172 the corrected image file. If the user likes the color corrected image 
file, he may choose to create and access 176 a digital exhibit space, wherein 
the mentioned digital exhibit space is linked to the web site. The color 
corrected image file could also be uploaded 178 to digital exhibit space for 
10 viewing instead of printing. The user can then logoff to stop the viewing 180. A 
copy of the color corrected image file remains in the digital space created for 
the user (FIG. 4A) for use at a later time. 

To log on to the digital exhibit space to obtain access to digital color images 
15 (FIG. 4B) at some later time, the user starts the process 200 by accepting an 
offer to log on to the digital image space 202. It is determined then 208 if the 
user has a password, in which case he provides one 210, or needs to obtain 
one 214. As soon as a password is provided, the user can view the images he 
created in digital image space 212 and log off when finished to stop 212 the 
20 process. 

The presently preferred embodiment of the invention, therefore, provides a 
method of correcting colors in a digital image taken from an image generating 
device and a software that makes the color corrected images available as 
25 either a downloadable image file or simply a file for viewing in the digital 
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space. It is expected to benefit a novice photographer to a more experienced 
photographer in ternns of time, cost and quality of color print that is made. 

5 Although the invention has been described in detail with reference to 
particular preferred embodiments, persons possessing ordinary skill in the art 
to which this invention pertains will appreciate that various modifications and 
enhancements may be made without departing from the spirit and scope of 
the claims that follow. For example, a standard film cameras may incorporate 

10 software that identifies when a target is being imaged and use the camera's 
exposure control sensors to capture ambient lighting information. Such 
information is compared with profile information stored within the camera and 
a correction value can be printed to the film for use by a processing 
laboratory. Further, while a software implementation of the invention is 

15 described herein, It will be appreciated by those skilled in the art that the 
invention, or portions thereof, may comprise a mechanism that is 
implemented in hardware level logic. 
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